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Executive Summary 

At the request of the Office of Special Education and Student Services, the Office of Shared 
Accountability evaluated the Universal Design for Learning (UDL) projects during spring 2013. 
UDL is an instructional framework that seeks to give all students equal opportunities to learn, by 
providing multiple means of representation, of action and expression, and of engagement. As a 
pilot for implementing UDL within schools, Montgomery County Public Schools (MCPS) 
initiated the UDL Elementary School Project in 2010-2011 and the UDL Middle School Project 
in 2011-2012. Through these projects, a small team of staff at each selected school received 
customized professional development and consultation on UDL instructional practices. 

To inform future implementation efforts within MCPS, this study examined four questions: 
1) what was the extent of implementation of UDL instructional practices, 2) what was the impact 
of UDL implementation on student engagement, 3) what was the impact of UDL implementation 
on student’s independence in learning, and 4) what was the impact of UDL implementation on 
teaching practices? The study included the first three elementary and three middle schools in the 
project and collected data through classroom observations, student surveys, and staff surveys. 

Summary of Findings 

Question 1: Implementation of UDL Practices 

Based on observations at UDL project schools of 48 kindergarten-Grade 8 classes, just over one 
half of teachers consistently implemented UDL. They used at least four instructional practices 
that were 1) ways to present information other than verbally and with text or 2) ways for students 
to choose how to gain information or show what they know. This moderate level of 
implementation is reasonable given that the evaluation occurred two to three years into a five 
year project. More elementary (about two thirds) than middle school (just over one third) 
teachers had consistent implementation. 

Questions 2 & 3: Impact on Student Processes 

UDL implementation should positively impact two student processes: independence in learning 
(e.g., task initiation) and engagement. Three types of engagement were measured: academic 
(e.g., on task), affective (e.g., enthusiasm), and cognitive (e.g., self-regulation). Observations or 
surveys were used to measure these processes, depending on the grade level. There was evidence 
for positive impacts of UDL implementation on both student processes, with variations by grade 
level, type of process, and student subgroup. 

In 11 kindergarten-Grade 2 classes at UDL project schools, observers collected data on student 
processes using a single indicator for each type of engagement and two indicators for 
independence in learning. Based on these indicators, all or almost all students in the majority of 
classes demonstrated academic engagement, cognitive engagement, and independence in 
learning, but not affective engagement. 

Almost 1,500 students in Grades 3-8 at UDL project schools and at matched, comparison 
schools completed surveys on engagement; items on cognitive engagement also reflected 
independence in learning. The response rate was 93% for the student surveys. Live or six items 
were combined to form scales for each type of engagement; the scale ranged from 1-4 for 
academic and affective engagement and 1-5 for cognitive engagement. Lor Grade 3-5 students, 
affective engagement was significantly higher at UDL project schools (mean = 3.40) than at 
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comparison schools (mean = 3.28), but there were no differences for academic or cognitive 
engagement. These findings were largely the same for two student subgroups: English language 
learners and students with educational disabilities. For students in Grades 6-8, each type of 
engagement was significantly higher at project schools than at comparison schools: academic 
(mean = 3.15 vs. mean = 3.05), affective (mean = 2.94 vs. mean = 2.68), cognitive (mean = 3.50 
vs. mean = 3.33). These findings were not the same as the findings for the two subgroups. 

Question 4: Impact on Teacher Practices 

Forty teachers and media specialists at project schools completed an online survey about 
teaching practices that are key to UDF implementation: lesson planning for accessibility and 
teacher collaboration. The response rate for this survey was 82%. There was evidence for 
positive impacts of UDF implementation on these teaching practices. 

Fesson planning for accessibility requires front-loading, meaning the teacher incorporates UDF 
strategies during the creation of instruction and assessments, instead of adjusting lessons or 
assessments afterwards. Most survey respondents were front-loading their planning, as indicated 
by responses to two questions. First, since their involvement with the UDF project, about two 
thirds of respondents reported they had “completely changed” or “changed a lot” their use of 
individual planning time. Second, more than one half of respondents described these changes to 
their planning in terms of key UDF practices, such as building in student choices. Fesson 
planning for accessibility also requires building student choice throughout the school year. More 
than one half of respondents described a plan to build student choice, typically an informal plan. 

Teacher collaboration to support UDF focuses on discussions with other teachers about students 
and student choice and on materials that enhance accessibility. Among survey respondents, 
more than one half had discussions with other teachers within their grade level at least a few 
times a month about student learning preferences and about building student choice. Most 
responding teachers shared responsibility for creating materials that enhance accessibility. 
About one half of respondents indicated that they frequently or regularly work with other 
teachers to create such materials and regularly add them to a shared electronic folder at their 
school. Further, the majority of respondents reported that they knew how to access this 
electronic folder and checked this folder for materials when planning a lesson. However, less 
than one half of respondents reported collaboration across content areas or grade levels. 

Summary of Recommendations 

Based on the findings, program staff should focus training, guidance, and other supports on the 
following suggestions to improve future implementation of the UDF projects: 

• Increase the number of UDF practices implemented by encouraging more use of the 
following practices by all teachers: choices of products, choices of responses, variety of 
formats for handouts, and reflection about choice. 

• Increase the number of UDF practices implemented by encouraging more middle school 
teachers to incorporate stations/centers and routines for making choices. 

• Encourage teachers to create formal plans on how they will build students’ abilities to make 
choices throughout the school year. 

• Increase the sharing of materials that enhance accessibility with teachers in other content 
areas and across grade levels. 
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Evaluation of the Universal Design for Learning Projects 

Elizabeth Cooper-Martin, Ph.D. and Natalie Wolanin 


Background 

Universal Design for Learning (UDL) is a set of principles for curriculum development that 
seeks to give all students equal opportunities to learn (National Center on UDL, 2012). UDL 
guides the creation of goals, methods, materials, and assessments so that these components of 
instruction can work for every student. Rather than being a “single, one-size-fits-all” approach, 
each component should be flexible with options for each learner in order to optimize every 
student’s learning (Meyer & Rose, 2005). 

The UDL framework is based on recent brain research that has identified three networks related 
to learning: recognition (what we learn), strategic (how we learn), and affective (why we learn) 
(Meyer & Rose, 2005). UDL’s three guiding principles call for flexibility and multiple methods 
to address individual differences related to each network and its area of specialty, as follows: 

• Principle I — Provide multiple means of representation to support the recognition 
networks for receiving and analyzing information. 

• Principle II — Provide multiple means of action and expression to support the strategic 
networks for planning and executing actions. 

• Principle III — Provide multiple means of engagement to support the affective 
networks for evaluating and setting priorities. 

Three guidelines and multiple checkpoints support each UDL principle (Ligure 1). By using 
these principles and guidelines to design curriculum and lessons, teachers can appropriately 
challenge, support, and engage all learners (Meyer & Rose, 2005). This framework was 
designed to meet the increased diversity of students in schools and to take advantage of the 
flexibility provided by computers and digital media. The latter can provide links for additional 
challenge or support, display content in various forms, and alter the presentation of content. 
However, there are many no-technology and low-technology ways to implement UDL principles 
(Maryland State Department of Education [MSDE], 2011). 

To infuse UDL into school practice, the Office of Special Education and Student Services 
(OSESS) initiated the UDL Middle School Project in 2010-2011 and the UDL Elementary 
School Project the following school year. The purpose of both projects is to develop a more 
comprehensive knowledge base and scalable framework for school-level implementation of UDL 
within Montgomery County Public Schools (MCPS). 

Both UDL projects support MCPS’s mission that every student will have the academic, creative 
problem solving, and social emotional skills to be successful in college and career (MCPS, 
2013). 
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Figure 1. Universal Design for Learning Guidelines. 
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Further, UDL supports the following actions that MCPS staff are expected to deliver, in order to 
support the district’s mission (MCPS, 2013): 

• Differentiate instruction to meet the needs of all students. 

• Evaluate what students know and are able to do through multiple and diverse 
measures. 

• Engage students in active learning of relevant and challenging content. 

• Create an environment that fosters student learning in a variety of ways and settings. 

• Empower students to take ownership of learning. 

Program Description 

Background, Goals, and Participants 

The UDL Elementary School and Middle School projects are based on the premise that 
implementation is more effective when initiated by a small, focused group within a school and 
then disseminated schoolwide (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; McREL, 
2008). Therefore, the project focus is on building a group of leaders, known as the Instructional 
Technology Leadership Team (ITLT). At each school, the ITLT includes 7 to 12 staff members. 
They are expected to become mentors and coaches on UDL strategies and technology for the 
benefit of all staff and students at their school (Collette & Richardson, 2010, 2011). 

The UDL Elementary School and Middle School Projects share the same goals, as follows: 

• Identify models for schools to develop leadership teams focused on schoolwide 
implementation of UDL over a three- to five-year period. 

• Identify scalable models to guide the professional learning of UDL leadership teams 
across multiple schools. 

• Clarify qualitative and quantitative data collection tools to measure changes in 
instructional practices related to UDL, guide professional learning needs assessment, 
and measure schoolwide changes. 

• Develop school- and district-level models for teachers to share UDL practices, 
materials, and technology tools, and how UDL impacts student learning. 

• Support curriculum development, purchase of instructional materials, and technology 
integration to best support UDL countywide. 

• Create web-based, on-demand resources for teachers and administrators in MCPS and 
throughout the country to support school-level UDL implementation. 

Schools had to apply to participate in each project (Collette & Richardson, 2010, 2011). Based 
on their applications, three elementary schools, Forest Knolls, Great Seneca Creek, and Sargent 
Shriver, were chosen for the UDL Elementary School Project, and both Lakelands Park and Rosa 
M. Parks middle schools were selected for the UDL Middle School Project. Because MCPS had 
identified Tilden Middle School as the site of the Middle School Special Education Institute, 
Tilden was selected as the third school for the UDL Middle School Project. 
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Supports Provided 

At each school, ITLT members received customized training and support on UDL and 
instructional technology. Each ITLT member who was a classroom teacher received a digital 
camera to enhance instruction and capture classroom implementation examples. To enhance the 
development of lesson plans, participating schools also received equipment such as computing 
devices (e.g., netbooks), headsets, audio splitters, and digital cameras. 

Program staff members, who support both projects, are members of the High Incidence 
Accessible Technology (HIAT) team, within the Physical Disabilities Program unit of OSESS. 
HIAT is a collaborative team that applies the principles of UDL to support school teams to meet 
the needs of all students (HIAT, 2012). 

Middle School Project 

The project for middle schools started in school year 2010-2011. During the first year, each 
ITLT member completed a one-credit, 15-hour course, taught by MCPS staff members, in 
summer or fall 2010. The ITLT met every two weeks for 60-90 minutes during the school year. 
The meetings focused on implementing UDL, using technology and no-technology supports to 
implement UDL, and defining processes to develop the ITLT as a professional learning 
community. A program staff member facilitated the meetings and visited each school at least 
weekly to assist with the development of lesson plans by visiting classes or attending teachers’ 
planning periods. 

During the second year, 2011-2012, the ITLT focused on providing professional development 
and supports to other instructional staff members (including paraeducators) at their school and 
improving their own skills at implementing UDL. Each pilot school also hosted seventh grade 
co-teaching teams from other schools who came to observe UDL implementation. During school 
year 2012-2013, ITLT members continued activities as a professional learning community. 

Elementary School Project 

Likewise, during the first year of the project at elementary schools, 2011-2012, ITLT members 
participated in professional development including a three-credit (45 hour) online Continuing 
Professional Development course. The course, which was developed by MCPS, focused on 
technology for implementing UDL and on leadership and coaching to support the use of UDL 
throughout the school. Additionally, the ITLT met monthly; members planned the meetings in 
consultation with program staff members as needed. HIAT staff members visited each school 
monthly to assist with the development of lesson plans and observe classes. During school year 
2012-2013, each ITLT continued activities as a professional learning community. 

Methods of Intervention 

As a result of professional development trainings and opportunities provided through the UDL 
projects, ITLT members at both elementary and middle schools are expected to implement UDL 
in their instruction. Program staff identified instructional practices that embody the principles of 
UDL and, based on initial work with ITLTs, represented changes from existing practices for 
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teachers. Therefore, each of their lessons should include at least some of the following 
instructional practices, centered on student choice and flexibility in teacher presentation. 

Student Choice 

To support and challenge the diverse learning styles of students, a teacher should provide them 
with choices in how they gain information and show what they know. These choices would be in 
one or more of the following areas: 

1. Products. The teacher provides students with choices for responses and products that 
demonstrate their skill and knowledge (e.g., verbal, written, drawing, physical 
demonstration, technology). 

2. Tools. The teacher provides students with choices for types of tools to generate products 
that demonstrate their skill and knowledge (e.g., paper-pencil, computer, Promethean 
Board, alternatives to handwriting, calculator). 

3. Stations/centers/groups. The teacher provides a variety of choices in methods to learn 
information that align with diverse learning styles (e.g., technology, readings at varied 
levels). 

4. Routines. The teacher establishes expectations, procedures, and routines that allow 
students to be reasonably independent with respect to choices or options in learning tools, 
materials, or methods (e.g., transition to stations, use of technology). 

Flexibility in Teacher Presentations 

The teacher should present information using multiple methods to complement text and verbal 
presentations in order to support and challenge diverse learning styles. These methods would be 
in one or more of the following areas: 

1. Curriculum materials. The teacher presents materials in additional formats beyond 
viewable text and a teacher speaking (e.g., text in digital files that could be read aloud, 
online resources, audio, video, pictures, charts). 

2. Explanatory devices. The teacher uses multiple types of explanatory devices (e.g., 
concept maps, graphic organizers, demonstration, pictures, audio/video, written, 
diagrams, charts, models, manipulatives). 

3. Drawings or images. The teacher uses drawings or images in paper handouts, digital 
materials, and presentations to complement text and a teacher speaking. 

Project Outcomes for Teachers 
Lesson Planning for Accessibility 

To create instructional materials and strategies that reflect UDL guidelines, teachers are expected 
to incorporate UDL strategies during development of curriculum, instruction, and assessments, 
instead of adjusting them afterwards to meet the needs of individual students (Delaware 
Department of Education [DDE], 2004; MSDE, 2011). In other words, teachers are expected to 
“front-load” lesson planning rather than retrofit. Further, because UDL practices encourage 
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students to select materials, tools, and products, teachers need to develop skills and routines that 
build each student’s ability to make choices that best support his/her learning (MSDE, 201 1). 

Teacher Collaboration 

Collaboration among teachers can reduce the need for each one to create instructional materials 
and strategies that incorporate UDL principles (MSDE, 2011). Therefore, teachers involved in 
the UDL projects are expected to share responsibility for creating materials and share digital 
materials and other resources. There should be a consistent, understandable, and schoolwide 
organization of electronic folders, and all staff should know where to look for resources. Along 
with the logistical setup, both dialogue and problem solving needs to happen to encourage the 
spread of sharing practices within and across content areas or grade levels. Additionally, 
teachers should discuss student learning profiles to ensure access to all students. Lastly, teachers 
should work together to build students’ capacity for making choices (i.e., learning tools, tasks, or 
products) throughout the school year and across grades. 

Project Outcomes for Students 

As a framework for curriculum design, instructional processes, and assessment, UDL strives to 
provide equal opportunities for every student to access the curriculum and to demonstrate what 
he/she has learned (MSDE, 2011). Therefore, all students benefit from implementation of 
UDL — including those who are gifted and talented; English language learners; students without 
disabilities; as well as students with physical, cognitive, or emotional disabilities. 

Engagement 

One way that students benefit from UDL is higher levels of engagement with school. When 
teachers provide multiple means of engagement, students are expected to be more involved and 
invested in learning. When students are challenged and also receive support to reach those 
challenges, school is more rewarding (DDE, 2004). Implementation of UDL frequently includes 
technology to provide choices to students and support their learning; an additional benefit of 
such technology is that many students are more engaged in schoolwork when they can use 
technology (DDE, 2004). 

Independence in Learning 

To take advantage of the flexibility and choice offered by UDL, students must make choices. 
Lor example, a teacher may use digital text to offer options for perception, but the student needs 
to select the options that best support his/her learning, such as font size, sound, images, or 
highlighting of main ideas (Meyer & Rose, 2005). Therefore, students need to develop the 
ability to make choices among materials, tools, and products (MSDE, 2011). Thus, another 
expected benefit of UDL is that students will become more independent in their learning. 

Student Learning 

UDL’s ultimate goal is to improve student learning. To achieve this outcome, teachers must 
consistently implement UDL with fidelity, and that implementation must have the expected 
impact of increased student engagement and greater independence in learning. If students are 
more engaged and more independent in their learning, their learning is expected to increase. 
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Evaluation Questions 

This evaluation examined the fidelity and extent of implementation of UDL and its impact on the 
attitudes and behaviors of students and on teaching practices. It was important to confirm that 
teachers are implementing UDL and that such implementation is having the expected impact on 
students and teachers, prior to determining UDL’s effect on student learning. The goal was to 
provide feedback to the program staff to support development of an effective model of classroom 
and school-level implementation of UDL that is scalable to a large district. 

1. Do teachers who received UDL professional development and consultation consistently 
implement UDL? Areas to examine include the following: 

a. Student choices 

i. Products: To what extent do teachers provide choices to students for responding 
and products that demonstrate their skill and knowledge? 

ii. Tools: To what extent do teachers provide choices for types of tools to generate 
products that demonstrate students’ skill and knowledge? 

iii. Stations/centers/groups: To what extent do teachers provide choices in methods to 
learn information that tap into diverse learning styles? 

iv. Routines: To what extent do teachers establish expectations, procedures, and 
routines related to student choices and options in learning tools, materials, and 
methods? 

b. Flexibility in teacher presentations 

i. Curriculum materials: To what extent do teachers present materials in additional 
formats beyond viewable text and the teacher speaking? 

ii. Explanatory devices: To what extent do teachers use multiple types of 
explanatory devices? 

iii. Drawings or images: To what extent do teachers use drawings or images in paper 
handouts, digital materials, and presentations to complement text and the teacher 
speaking? 


2. What is the impact of UDL implementation on student engagement? Areas to examine 
include the following: 

a. What is the impact of UDL implementation on the following forms of engagement? 

i. Academic, defined as time on task during class and homework completion 

ii. Affective, defined as enthusiasm, excitement, and enjoyment of class 

iii. Cognitive, defined as self-regulation and being strategic about class work 

b. Is the impact of UDL implementation on student engagement consistent for each of 
the following student groups? 

i. English language learners 

ii. Students with educational disabilities 
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3. What is the impact of UDL implementation on student independence in learning processes? 
Areas to investigate include the following: 

a. What is the impact of UDL implementation on learning processes, such as the 
following? 

i. Task initiation 

ii. Self- selection of learning tools, tasks, or products 

iii. Independence in completing activities 

iv. Self-monitoring of task completion 

b. Is the impact of UDL implementation on student independence in learning processes 
consistent for each of the following student groups? 

i. English language learners 

ii. Students with educational disabilities 

4. What is the impact of UDL implementation on lesson planning for accessibility and teacher 
collaboration? Areas to investigate include the following: 

a. Lesson planning for accessibility 

i. Front-loading lesson planning 

ii. Planning for building student choice throughout the school year 

b. Teacher collaboration practices 

i. Discussion of student learning profiles 

ii. Discussion of building student choice across grades 

iii. Sharing of responsibility to create materials that enhance accessibility 

iv. Knowledge to access already-created materials that enhance accessibility 

v. Sharing materials across content areas or grade levels 
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Methodology 

To answer all evaluation questions, this study utilized a multimethod data collection strategy, 
including classroom observations, student surveys, and staff surveys. Based on program 
materials and previous research studies, the evaluators, in collaboration with staff members from 
the Physical Disabilities Program and the HIAT team, developed instruments for these activities. 

This study included the three middle schools that joined the UDL project in 2010-2011: 
Lakelands Park, Rosa M. Parks, and Tilden, plus the three elementary schools that joined the 
UDL project in 2011-2012: Forest Knolls, Great Seneca Creek, and Sargent Shriver. To 
accommodate this age range, different methods were used to collect data on the same construct. 

Data Collection Activities 
Classroom Observations 

Instrument. Program staff members had developed an observation protocol for 
classroom visits which included the four categories of student choices and the three categories of 
teacher flexibility listed in Evaluation Question 1. Through discussions between evaluators and 
program staff and pilot observations, the protocol was refined and some categories were changed 
to insure reliable reporting. The final observation protocol included nine indicators, as listed and 
defined in Table 1. Each indicator had multiple options for observers to document the extent of 
evidence; options varied by indicator (see detail in Table 1). 

Table 1 


UDL Protocol Indicators, Definitions, and Options for Extent of Evidence 


Indicators of student 
choices 

Definition 

Options for 
extent of evidence 

Choices of responses 

Teacher provides students (in whole group or a small group) with 

None, evident. 


choices for responses when called on. 

Not applicable. 

Choices of products 

Teacher provides students with choices for products (e.g., student 

None, minimal, evident. 


work, informal assessments) that demonstrate their skill & 
knowledge. 

Not applicable. 

Choices of tools for 

Teacher provides students with choices for types of tools that they 

None, evident. 

production 

can use to produce/generate products, responses, or informal 
assessments that demonstrate their skill and knowledge. 

Not applicable. 

Choices of methods 

Teacher provides students with choices of methods of learning that 

None, minimal, evident. 

to learn information 

tap into diverse learning styles or offer different learning 
experiences. 


Stations/centers 

Teacher provides stations or centers. 

None, minimal, evident, 
evident with emphasis. 

Routines for making 

Teacher establishes expectations, procedures, and routines related to 

None, minimal, evident. 

choices 

student choices of products, tools, or methods. 

evident with emphasis. 
Not applicable. 

Reflection about 
choice 

Teacher asks students to reflect on their choice. 

None, evident. 

Indicators of teacher 


Options for extent of 

flexibility 

Definition 

evidence 

V ariety of 

Teacher uses additional formats beyond viewable text and a teacher 

None, minimal, evident. 

presentation formats 

speaking, when presenting to students as a whole group or a small 

evident with emphasis. 


group. 

Not applicable. 

V ariety of formats 

Teacher provides additional formats beyond hard copy with only 

None, minimal, evident. 

for handouts 

text. 

Not applicable. 
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Options for extent of evidence were defined as follows. Minimal meant that there was some 
evidence of the practice, but that the teacher did not fully implement it as expected by the 
program staff. Evident meant that the teacher implemented the practice as expected. Evident 
with emphasis meant that the teacher implemented the practice to a degree that was distinctly 
better than the expected level, as defined by program staff. 

In kindergarten-Grade 2 classes, observers also collected data on student processes. If the 
teacher assigned a task to the students, observers collected data on students’ independence in 
learning processes, using two indicators: 1) Students start work promptly without prodding by 
teacher; and 2) Students work independently (without consulting teacher) to complete the 
activity. 

Observers collected data on engagement two times during each kindergarten-Grade 2 class, 
using a single indicator for each type of engagement, as follows: 

• Academic: Students are on task. 

• Affective: Students show enthusiasm. 

• Cognitive: Students are self-regulated. 

Observable evidence for each of the indicators was as follows. For academic engagement, two 
types of evidence were required: 1) students do the assigned work and 2) students do not engage 
in off-task behavior (e.g., get out of seat without permission, have off-topic conversation, gaze 
out window). For affective engagement, at least one of the following types of evidence was 
required: 1) students vocalize/express excitement about content/activities (e.g., “oohs & aahs”); 
2) students want to take a turn; 3) students eagerly raise their hands; or 4) students want to 
participate. For cognitive engagement, there was one type of evidence: that students avoid 
disciplinary encounters. 

For each indicator of engagement and independence in learning in kindergarten-Grade 2 classes, 
the observer recorded how many students demonstrated the indicator using the following 
categories: all, almost all, most, some, few, none. 

Sample. The observation sample included all teachers and media specialists on the ITFT 
team at each UDF project school in the sample. The total was 25 teachers and media specialists 
at elementary schools and 25 teachers and media specialists at middle schools. 

Data Collection. Four staff members from the Office of Shared Accountability (OS A) 
were trained in using the protocol, with support from staff members from the Physical 
Disabilities Program and the HIAT team. Each observer visited only elementary or only middle 
schools. The observation was one class period (about 45 minutes) for each middle school 
teacher or media specialist and a comparable length of time (i.e., 40-45 minutes) for each 
elementary school teacher or media specialist. Observations occurred between April 15 and May 
2, 2013. All 25 elementary observations were completed. Observations of two middle school 
teachers could not be scheduled; thus 23 (92%) middle school observations were completed. 
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Student Surveys 

Sample. The student survey sample included all students in observed classes in Grades 
3-8. To provide a comparison group of students, six comparison schools were selected based on 
similarity to UDL project schools. Schools eligible to be a comparison school included those 
that were not participating in and had not applied to participate in the UDL project, had not 
received extensive UDL training from MCPS program staff, and did not mention UDL in their 
School Improvement Plan. Advanced statistical analysis identified the three eligible schools 
most similar to each UDL project school in the study, based on the following variables: 

1. Percentage of Black or African American students 

2. Percentage of Asian American students 

3. Percentage of Hispanic/Latino students 

4. Percentage of White students 

5. Percentage of students receiving English for Speakers of Other Languages (ESOL) 
services 

6. Percentage of students receiving Free and Reduced-price Meal System (FARMS) 
services 

7. Percentage of students receiving special education services 

8. Percentage of special education students who spend 80% or more of their instructional 
time in general education classes 

9. Percentage of special education students who spend at least 40% but less than 80% of 
their instructional time in general education classes 

10. Percentage of special education students who spend less than 40% of their instructional 
time in general education classes 

11. Average class size (i.e., for Grades 4-5 in elementary schools, for English class in middle 
schools, and for all classes other than English in middle schools) 

Program staff used their knowledge and experience with schools to make the final selection of 
one comparison school for each UDL project school in the sample. Next, within each 
comparison school, one of the authors selected a comparable classroom to match each observed 
class, in terms of grade level and subject area. The comparable classroom had to be the same 
grade level as the observed classroom. Therefore, the sample excluded students of media 
specialists, due to the difficulty of identifying a class receiving instruction in the media center at 
the comparison school at the same grade level as the observed class during the survey 
administration window. The final sample of students included 661 in Grades 3-5 and 934 in 
Grades 6-8. 

Instrument. The student survey items concerned three types of engagement: academic, 
affective, and cognitive. All engagement items referred to the specific class that the student was 
attending at the time of the survey. The items for cognitive engagement also reflected 
independence in learning processes; these items concerned self-regulation and being strategic 
about class work. Items on academic engagement referred to time on task during class and 
homework completion, and those about affective engagement concerned enthusiasm, excitement, 
and enjoyment of the class. The authors modified items used in previous surveys on engagement 
of elementary or middle school students for this study (Appleton, Christenson, Kim, & Reschly, 
2006; National Center for School Engagement, 2006; Skinner, Kindermann, & Furrer, 2009). 
Items and response options for all three types of engagement are in Table 2. 
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Table 2 

Engagement Items on Student Survey, Response Options, and Cronbach’ 

s Alpha Values 

Engagement items 

Response options 
(value for scale) 

Cronbach’s alpha 3 
Grades Grades 

3-5 6-8 

Academic 

• When I’m in this class, 1 think about other things, (reverse 
coded) 

• In this class, I work as hard as I can. 

• When I’m in this class, I listen very carefully. 

• When I’m in this class, 1 just act like I’m working, even 
though I’m not. (reverse coded) 

• I try very hard to do well on my homework for this class. 

• I pay attention in this class. 

Very true (4) 

Sort of true (3) 
Not very true (2) 
Not at all true (1) 

.76 

.82 

Affective 

• When I’m in this class, 1 feel good. 

• When we work on something in this class, I feel interested. 

• I feel excited by the work in this class. 

• I enjoy the work I do in this class. 

• This class is fun. 

• I enjoy learning new things in this class. 

Very true (4) 

Sort of true (3) 
Not very true (2) 
Not at all true (1) 

.87 

.90 

Cognitive 

• When I read something for this class, I ask myself 
questions to make sure I understand what it is about. 

• I check my classwork for mistakes. 

• If 1 don’t understand what I read for this class, I go back 
and read it over again. 

• After I do homework for this class, I look it over to see if 
it’s correct. 

• If 1 don’t know what a word means when 1 am reading for 
this class, I try to find out. 

Always (5) 
Often (4) 
Sometimes (3) 
Seldom (2) 
Never (1) 

.72 

.76 


“Cronbach’s alpha measures the scale’s internal reliability; values >.70 are considered acceptable (Nunnaly, 1978). 


Scales. For better reliability, a scale of multiple items, rather than a single item, was used 
to measure each type of engagement, as shown in Table 2. The internal consistency or reliability 
of each scale was calculated and examined in this study using Cronbach’s alpha (Nunnaly, 
1978). The internal reliability was considered satisfactory because Cronbach’s alpha was at least 
.70 for each scale (Table 2). 

Administration. An OS A staff member administered the survey to students in their 
classrooms between May 1 and May 23, 2013. Students completed a hard copy of the survey. 
The survey administrator read the directions, which stated that students’ responses would be 
confidential, that their teachers would not see their responses, and that information would be 
reported only at the aggregate level. The survey administrator also read each item and each 
response option out loud. Survey administrators read information out loud to ensure student 
understanding and to provide another presentation format. The overall response rate was 93%, 
including 626 respondents in Grades 3-5 for a 95% response rate, plus 862 respondents in 
Grades 6-8 for a 92% response rate. 
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Staff Surveys 

Sample. The sample for the staff survey was the same as the observation sample; it 
included all teachers and media specialists on the ITLT team at each UDL project school. The 
total was 25 at elementary schools and 25 at middle schools. 

Instrument. The staff survey was online and focused on lesson planning for accessibility 
and collaboration with other teachers. It included questions on individual and team lesson 
planning; planning for building student choice throughout the school year individually and with 
other teachers; accessing, creating, and sharing materials that enhance accessibility; and 
discussion of student learning profiles. 

Administration. On May 28, 2013, one of the authors sent an e-mail with a link to the 
survey to each member of the sample, followed by three e-mail reminders during June. E-mails 
went to 49 staff members; due to an oversight, one middle school teacher did not receive the 
e-mails. Forty staff members completed the survey for an overall response rate of 83%. The 
response rate among elementary school staff members was 88% (22 of 25) and among middle 
school staff members was 75% (18 of 24). 

Analytical Procedures 
Evaluation Questions 1 and 4 

Descriptive statistics were used to analyze the findings for Evaluation Question 1 on teacher’s 
implementation of UDL practices and for Evaluation Question 4 on teacher’s lesson planning for 
accessibility and collaboration with other teachers. To test for significant differences in teaching 
practices between elementary and middle school students, f - tests were used. 

Evaluation Questions 2 and 3 

Evaluation Questions 2a and 3a concern the impact of UDL implementation on student 
engagement and independence in learning processes, respectively. Stepwise regression analysis 
was used for these questions, to test whether students in classrooms with UDL implementation 
had higher levels of engagement and independence in learning processes than students in 
classrooms at comparison schools. The measure of engagement equaled the mean value across 
all scale items (using the scale values in Table 2) for each student who answered all items or all 
but one item in the scale. The regression analyses controlled for differences in the following 
student characteristics: grade level, course subject, gender, race and ethnicity, receipt of FARMS 
services, receipt of special education services, receipt of ESOL services, number of tardies, 
number of suspensions, and grade point average (only for students in Grades 6-8). There were 
separate analyses for each type of engagement and elementary versus middle school students. 

To answer Evaluation Questions 2b and 3b, on whether the impact of implementation of UDL is 
consistent for different student groups, the regressions for Questions 2a and 3a were repeated to 
include only English language learners and only students with educational disabilities. English 
language learners were defined as students currently receiving ESOL services plus students who 
had recently exited from these services. For Grades 3-5, English language learners included 
students who were currently receiving or had ever received ESOL services. For Grades 6-8, 
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English language learners included students currently receiving ESOL services plus those who 
had exited ESOL within the last two years. For all grades, students with educational disabilities 
included students currently receiving special education services. If the number of students in a 
subgroup was too small (i.e., < 100) for regression analysis, t-tests were used. Unlike regression, 
t-tests cannot control for differences between students. 

For all analyses, tests of statistical significance were calculated to judge whether the observed 
relationship between UDL implementation and student processes (e.g., engagement) occurred by 
chance. Also, tests of practical significance were calculated to judge whether the observed 
relationships were large enough to be useful to program staff (American Psychological 
Association, 2001). Effect sizes were used as tests of practical significance. 

For regression analyses, standardized regression coefficients ((3 values) were used as an effect 
size measure (Kline, 2005). To interpret the magnitude of P values, the following guidelines 
from Cohen (1988) were used: .10, .30, and .50 which correspond to small, medium, and large 
effect sizes, respectively. 

For t-tests, Cohen’s d was used to measure the effect size. Cohen (1988) proposed the following 
guidelines for d: .20, .50, and .80 correspond to small, medium, and large effect sizes, 
respectively. Cohen’s d was calculated as follows: (MeanuDL - Mean c )/SD A LL (Vacha-Hasse & 
Thompson, 2004). In this equation, MeanuDL was the mean of the responses from students at 
UDL project schools, Meanc was the mean of the responses from the students at comparison 
schools, and SD A ll was the standard deviation of the responses from students at all schools. 

Strengths and Limitations of the Methodology 

One strength of this study is that all schools involved in the first year of both the elementary and 
middle school UDL projects participated in the evaluation. Further, to measure the extent of 
implementation of UDL practices, classroom observations were conducted to capture teacher 
behavior. An additional strength was not specifying to the observed teacher which day or time 
the observer would arrive; this approach lessened the possibility that the observed teacher would 
change behaviors to meet the observer’s expectations. Also, the observation protocol was 
developed by working closely with program staff members who are experts on UDL 
implementation. Lastly, the high response rates to the student surveys (93%) and staff survey 
(82%) suggest that the survey responses reflect the experiences of most students and staff 
members in the samples. 

One limitation of this study is the use of self-reports from staff surveys to evaluate certain 
aspects of UDL implementation; it is possible that these responses were self-serving. Another 
limitation was the use of observations to measure student engagement and independence in 
learning processes for students in kindergarten-Grade 2. These processes are best measured by 
asking the student questions because these processes are internal to the student. However, this 
approach was not possible for these students; they were too young to reliably complete written 
surveys and the sample size was too large for individual interviews. Additionally, observers 
could not distinguish which students were English language learners or which ones had 
educational disabilities; therefore, it was not possible to analyze whether the findings about 
engagement and independence in learning processes were consistent for these subgroups among 
students in kindergarten-Grade 2. 
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Results 


Findings for Question 1 

Do teachers who received UDL professional development and consultation consistently 
implement UDL? 

Level of Implementation 

Data from classroom observations were used to evaluate implementation of UDL by project 
teachers, including media specialists, at all grade levels. The observation protocol included nine 
UDL instructional practices. Program staff members agreed that evidence of at least four UDL 
practices during an observed class was consistent (i.e., adequate) implementation. Two 
approaches to measuring the evidence were used. 

Noncompensatory . The first approach was noncompensatory; it required that each 
practice had to be at the evident or evident with emphasis level. (Evident meant that the teacher 
implemented the practice as expected and evident with emphasis meant that the teacher 
implemented the practice to a degree that was distinctly better than the expected level, as defined 
by program staff.) With this approach, consistent implementation of UDL was at a low level 
across all project schools; only 15 teachers, just under one third of those observed (31%), 
demonstrated four or more UDL practices (Table 3). This total of 15 included 9 elementary 
teachers (36%) and 6 middle school teachers (26%) 

Table 3 


Number of Observed UDL Instructional Practices for All Teachers, by School Level 



ii > 

oo 

Elementary school 
(n = 25) 

Middle school 
(n = 23) 

n 

% 

n 

% 

n 

% 

# UDL practices noncompensatory 3 

0 to 3 

33 

69 

16 

64 

17 

74 

4 to 7 

15 

31 

9 

36 

6 

26 

# UDL practices compensatory b 

0 to 3 

23 

48 

9 

36 

14 

61 

4+/- to 7 

25 

52 

16 

64 

9 

39 


a All practices were at the evident or evident with emphasis level. 

b 4+/- could include two practices at the evident level, one practice at the evident with emphasis level, 
and one practice at the minimal level. 


Compensatory. The second approach to measuring the evidence was compensatory; 
teachers that met the standard of four practices included those with two practices at the evident 
level, one practice at the evident with emphasis level, and one practice at the minimal level. 
(Minimal meant that there was some evidence of the practice, but that the teacher did not fully 
implement it as expected.) With this approach, the practice at the evident with emphasis level 
compensated for the practice at the minimal level. This level was labeled 4+/- to indicate that 
one practice could be at the “evident with emphasis level” (+) and one at the “minimal level” (-). 
With this approach, consistent implementation of UDL was at a moderate level across all project 
schools; about one half of all observed teachers (52%) demonstrated consistent implementation 
(Table 3). This total included nearly two thirds (64%) of the observed elementary teachers and 
more than one third of the observed middle school teachers (39%). 
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Implementation of Specific Practices 

All Teachers. Among all observed teachers, about one half had evidence of two 
indicators: variety of presentation formats (56%) and choice of tools for production (48%) (Table 
4). Although one half of observed elementary teachers provided choices of methods to learn 
information (52%) and had established routines for making choices (52%), less than one third of 
observed middle school teachers showed evidence of these two practices. Further, although no 
middle school teachers used stations/centers, about one half (48%) of all elementary teachers did, 
including 50% of kindergarten-Grade 2 teachers and 45% of Grades 3-5 teachers. Close to one 
half of all middle school teachers (44%) used a variety of formats for handouts, but only one 
elementary teacher did so. Less than one quarter of all observed teachers, of elementary school 
teachers, and of middle school teachers were observed using the remaining practices: variety of 
formats for handouts, choices of products, choices of responses, and reflection about choice. 


Table 4 

Frequency of Specific UDL Instructional Practices Among All Teachers by School Level 


UDL practice 

(N 

All 
= 48) 

Elementary school 
(n = 25) 

Middle school 
(n = 23) 

n 

% 

n 

% 

n 

% 

V ariety of presentation formats 

27 

56 

ii 

44 

16 

70 

Choices of tools for production 

23 

48 

12 

48 

11 

48 

Choices of methods to learn information 

20 

42 

13 

52 

7 

30 

Routines for making choices 

18 

38 

13 

52 

5 

22 

Stations/centers 

12 

25 

12 

48 

0 

0 

V ariety of formats for hand outs 

11 

23 

1 

4 

10 

44 

Choices of products 

10 

21 

6 

24 

4 

17 

Choices of responses 

9 

19 

5 

20 

4 

17 

Reflection about choice 

4 

8 

3 

12 

1 

4 


Note. Teachers used more than one instructional practice. 


Teachers with 4+/- Implementation. Among the 25 teachers with implementation at the 
4+/- level or higher, the majority had evidence of four indicators: routines for making choices 
(17, 68%), choice of methods to learn information (16, 64%), variety of presentation formats (16, 
64%), and choice of tools for production (14, 56%) (Table 5). 

Table 5 

Frequency of Specific UDL Instructional Practices Among Teachers with UDL 


Implementation at the 4+/- 

■ level or Higl 

aer, by School Level 


All 

Elementary school 

Middle 

school 


(N = 

25) 

(n = 

16) 

(n = 

9) 

UDL practice 

n 

% 

n 

% 

n 

% 

Routines for making choices 

17 

68 

13 

81 

4 

44 

Choices of methods to learn information 

16 

64 

11 

69 

5 

56 

Variety of presentation formats 

16 

64 

8 

50 

8 

89 

Choices of tools for production 

14 

56 

8 

50 

6 

67 

Stations/centers 

11 

44 

11 

69 

0 

0 

Choices of products 

7 

28 

4 

25 

3 

33 

Choices of responses 

6 

24 

4 

25 

2 

22 

V ariety of formats for hand outs 

5 

20 

1 

6 

4 

44 

Reflection about choice 

4 

16 

3 

19 

1 

11 


Note. Teachers used more than one instructional practice. 
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There were differences between school levels among teachers with implementation at the 4+/- 
level or higher (see Table 5 above). Evidence of routines for making choices was more common 
among elementary (13, 81%) than among middle school teachers (4, 44%). Similarly, the 
majority of elementary teachers with implementation at the 4+/- level or higher used 
stations/centers (13, 81%), but no middle school teachers did. However, while almost all middle 
school teachers with implementation at the 4+/- level or higher used a variety of presentation 
formats (8 of 9), only one half of the elementary teachers with implementation at the same level 
(8 of 16) did so. 

As with all observed teachers, there was evidence of the last four practices in Table 5 among 
relatively few teachers with implementation at the 4+/- level or higher. 

Findings for Questions 2 and 3 

What is the impact of UDL implementation on student engagement and on student independence 
in learning processes? 

The findings on the relationship between UDL implementation and the student processes of 
engagement and independence in learning are presented by grade level, because the evaluation 
methods varied by grade level. Academic engagement was defined as time on task and 
homework completion. Affective engagement was defined as enthusiasm, excitement, and 
enjoyment. Cognitive engagement was defined as self-regulation and being strategic. 
Independence in learning was defined as task initiation, independence in completing activities, 
and self-monitoring of learning. The expectation was that UDL implementation would have a 
positive effect on student engagement and independence in learning processes. 

Kindergarten-Grade 2 

Lor kindergarten-Grade 2, data on engagement and independence in learning came from 11 
classroom observations at UDL project schools. Lor each indicator of engagement and 
independence in learning, the observer recorded how many students demonstrated the indicator 
using the following categories: all, almost all, most, some, few, none. 

Engagement. Observers collected data on each type of student engagement twice for 
each class. The level of engagement varied by type of engagement (Table 6). 


Table 6 

Evidence for Student Engagement by Type of Engagement for Kindergarten-Grade 2 


All or almost all students engaged 


# classes ( N = 

11) 

Academic 

Affective 

Cognitive 

Observed both times 

6 

2 

10 

Observed one time 

3 

3 

0 

Not observed 

2 

6 

1 


Academic engagement was fairly high in the kindergarten-Grade 2 classes. Out of 11 classes, 
there were 6 in which all or almost all students were academically engaged at both observation 
times. There were three additional classes in which all or almost all students were academically 
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engaged at one of the observation times. There were only two classes in which observers did not 
see all or almost all students academically engaged at either observation time. 

Affective engagement was not consistently high in the observed classes. There were only two 
classes in which all or almost all students demonstrated affective engagement at both observed 
times and three additional classes in which all or almost all students demonstrated affective 
engagement at one of the observation times. 

Cognitive engagement was high in the observed classes. In 10 classes, all or almost all students 
demonstrated the indicator for cognitive engagement at both observation times. 

Independence in Learning Processes. For kindergarten-Grade 2, observers collected 
data on student independence in learning processes, using two indicators, whenever the teacher 
assigned a task for students to do on their own. Teachers assigned such a task in 8 of the 11 
classes. 

Student independence in learning processes was high in the majority of classes (Table 7). All or 
almost all students demonstrated both indicators of independence in learning processes in five of 
the eight classes. In one additional class, all or almost all students demonstrated one indicator. 


Table 7 

Evidence for Student Independence in 
Kindergarten-Grade 2 

Learning for 

All or almost all students demonstrated # classes ( N = 8) 

Both indicators 

5 

One indicator 

1 

None of the indicators 

2 


Student subgroups. Observers could not determine which students were English 
language learners or had educational disabilities. Therefore, it was not possible to analyze 
whether student processes were consistent for these subgroups of kindergarten-Grade 2 students. 
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Grades 3-5 

Students in Grades 3-5 at both project and comparison schools completed surveys about their 
levels of engagement (see Table 2). Survey items on cognitive engagement also measured 
independence in learning. 

Regression analysis was used to test for a significant relationship between UDL implementation 
and student engagement. If there were fewer than 100 responding students, t-tests were used 
instead of regression analysis. For all analyses, tests of practical significance were calculated to 
judge whether the observed relationships were large enough to be useful to program staff; effect 
sizes were used as tests of practical significance. 

Engagement. Among responding students, academic and cognitive engagement did not 
differ between UDL project and comparison schools, but affective engagement was higher at 
project schools (Table 8). 


Table 8 


Mean and Standard Deviation of Engagement Scales for Grades 3-5 

by School Group 


Type of engagement 
(range) 


Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic (1. 0-4.0) 

278 

3.42 

0.51 

287 

3.43 

0.46 

Affective (1. 0-4.0) 

279 

3.40 

0.56 

287 

3.28 

0.62 

Cognitive (1. 0-5.0) 

256 

3.80 

0.76 

267 

3.82 

0.77 


The relationship for affective engagement was statistically significant (P = .13, p < .01) 
(Table 9). This relationship also was practically significant with a small effect size (P > ■ 10), 
meaning that the difference in affective engagement between the two school groups was small, 
but large enough to be useful to program staff. The relationships for academic and cognitive 
engagement were not statistically or practically significant. 

Table 9 


The Relationship between UDL Implementation and Student Engagement, 
by Type of Engagement for Students in Grades 3-5 



Academic 
(N = 553) 

Affective 
(A =554) 

Cognitive 
( N= 572) 

Project school: B (SE) 

-0.03 (0.04) 

0.15 (0.05) 

-0.04 (0.07) 

Project school: P 

-0.04 

0.13** 

-0.03 

Model fit: F (df) 

9.86***(6) 

9.61***(5) 

5 76 ***(4) 

Model fit: adjusted 1C 

.09 

.07 

.04 


*p < . 05 , **p < . 01 , ***p < . 001 . 
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Student subgroups. Among responding students who were English language learners, 
academic and cognitive engagement did not differ between UDL project and comparison 
schools, but there was a small difference for affective engagement (Table 10). 


Table 10 

Mean and Standard Deviation of Engagement Scales for English 


Language Learners in Grades 3-5 by School Group 


Type of engagement 
(range) 

Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic (1. 0-4.0) 

115 

3.40 

0.52 

121 

3.40 

0.47 

Affective (1. 0-4.0) 

116 

3.42 

0.55 

121 

3.28 

0.63 

Cognitive (1. 0-5.0) 

107 

3.81 

0.70 

117 

3.84 

0.79 


For English language learners, the relationship for affective engagement was statistically 
significant ([3 = .16, p < .05) and practically significant with a small effect size (P > .10) 
(Table 11). 


Table 1 1 

The Relationship between UDL Implementation and Student Engagement, 
by Type of Engagement for English Language Learners in Grades 3-5 



Academic 
(N = 231) 

Affective 
(N = 232) 

Cognitive 
(N= 223) 

Project school: B (SE) 

-0.03 (0.06) 

0.19(0.08) 

0.00 (0.10) 

Project school: P 

-0.03 

0.16* 

0.00 

Model fit: F (df) 

6.01***(5) 

4 .42** (4) 

4.22*(2) 

Model fit: adjusted 1C 

.10 

.06 

.03 


*p < . 05 , **p < . 01 , ***p < . 001 . 


Because there were fewer than 100 responding students with educational disabilities, t-tests were 
used instead of regression analysis. For these students, academic engagement did not differ 
between project and comparison schools, nor did cognitive engagement (Table 12). Affective 
engagement was higher at project schools than at comparison schools. This difference was not 
statistically significant at conventional levels (t(58) = 1.38, p = .10), but was practically 
significant ( d = 0.45), with a small effect size (d > 0.20). 


Table 12 

Mean and Standard Deviation of Engagement Scales 
for Students with Educational Disabilities in Grades 3-5 by School Group 


Type of engagement (range) 


Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic ( 1 .0-4.0) 

19 

3.05 

0.76 

41 

3.19 

0.48 

Affective (1. 0-4.0) 

19 

3.41 

0.67 

41 

3.07 

0.77 

Cognitive (1. 0-5.0) 

18 

3.54 

0.88 

38 

3.47 

0.84 
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Consistency . Among responding students in Grades 3-5, the relationships between UDL 
implementation and student engagement for all students were very consistent with the 
relationships among English language learners and students with educational disabilities 
(Table 13). For all students, as for both subgroups, only affective engagement differed 
significantly between UDL project schools and comparison schools. This relationship was 
practically significant for all students and both subgroups and statistically significant for all 
students and English language learners. 


Table 13 


Summary of Significant Relationships between UDL Implementation and Student Engagement for All 
Students and Student Subgroups in Grades 3-5 by Type of Engagement 



All students 

Student subgroup 

English language learners 

Students with educational disabilities 

Type of 

Statistically 

Practically 

Statistically 

Practically 

Statistically 

Practically 

engagement 

significant 

significant 

significant 

significant 

significant 

significant 

Academic 

No 

No 

No 

No 

No 

No 

Affective 

Yes** 

Small 

Yes* 

Small 

No 

Small 

Cognitive 

No 

No 

No 

No 

No 

No 


*p < .05, **p < .01, ***p < .001. 


Independence in Learning Processes. The survey items about cognitive engagement 
also reflected independence in learning processes (see Table 2). The above findings indicated 
that there was not a relationship between UDL implementation and these processes for all 
students, for English language learners, or for students with educational disabilities (Table 13). 

Grades 6-8 

Data collection and analytical procedures for Grades 6-8 were the same as those described above 
for Grades 3-5. The results for middle school students follow. 


Engagement. Among responding middle school students, each type of engagement was 
higher at UDL project schools than at comparison schools (Table 14). 

Table 14 

Mean and Standard Deviation of Engagement Scales for Grades 6-8 


by School Group 


Type of engagement 
(range) 

Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic (1. 0-4.0) 

436 

3.15 

0.53 

375 

3.05 

0.60 

Affective (1. 0-4.0) 

437 

2.94 

0.67 

374 

2.68 

0.72 

Cognitive (1. 0-5.0) 

436 

3.50 

0.82 

374 

3.33 

0.82 


There was a statistically significant relationship for each type of engagement (Table 15): 
academic (P = .09, p < .001), affective (P = .11, p < .01), cognitive (P = .16, p < .001). The 
relationship also was practically significant for affective and cognitive engagement with small 
effect sizes (P > .10). 
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Table 15 


The Relationship between UDL Implementation and Student Engagement, 
by Type of Engagement for Students in Grades 6-8 



Academic 
(N = 763) 

Affective 
(A =762) 

Cognitive 
(N = 763) 

Project school: B (SE) 

0.10(0.04) 

0. (0.0) 

0. (0.0) 

Project school: P 

q Q9*** 

0.16*** 

0.11** 

Model fit: F (df) 

10.98***(6) 

8.23***(8) 

8.24***(7) 

Model fit: adjusted IE 

.07 

.08 

.06 


*p < . 05 , **p < . 01 , ***p < . 001 . 


Student Subgroups. Among English language learners in middle schools, each type of 
engagement was somewhat higher at UDL project schools than at comparison schools 
(Table 16). Because there were fewer than 100 responding students who were English language 
learners, t-tests were used instead of regression analysis. For academic engagement, the 
difference was not statistically significant (t(56) = 1.57, p > .05), but was practically significant 
( d = 0.42), with a small effect size (d > 0.20). Likewise, for affective engagement, the difference 
between project and comparison schools was not statistically significant (t(56) = 1.26, p > .05), 
but was practically significant ( d = 0.34), with a small effect size (d > 0.20). Lastly, the 
difference for cognitive engagement was both statistically (t(56) = 1.99, p = .05) and practically 
significant ( d = 0.53), with a medium effect size (d > 0.50). 

Table 16 


Mean and Standard Deviation of Engagement Scales for English 
Language Learners in Grades 6-8 by School Group 


Type of engagement 
(range) 


Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic (1. 0-4.0) 

37 

3.29 

0.59 

21 

3.05 

0.67 

Affective ( 1 .0-4.0) 

37 

3.28 

0.50 

21 

3.07 

0.78 

Cognitive (1. 0-5.0) 

37 

3.81 

0.76 

21 

3.36 

0.91 


Among students with educational disabilities, academic and affective engagement were 
somewhat higher at UDL project schools than at comparison schools, but cognitive engagement 
did not differ (Table 17). 


Table 17 

Mean and Standard Deviation of Engagement Scales for Students with 
Educational Disabilities in Grades 6-8 by School Group 


Type of engagement (range) 


Project schools 

Comparison schools 

n 

Mean 

Standard 

deviation 

n 

Mean 

Standard 

deviation 

Academic (1. 0-4.0) 

73 

3.19 

0.45 

72 

3.14 

0.60 

Affective ( 1 .0-4.0) 

73 

2.94 

0.71 

71 

2.78 

0.80 

Cognitive (1. 0-5.0) 

73 

3.43 

0.80 

71 

3.40 

0.89 
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None of the relationships between UDL implementation and student engagement were 
statistically significant for middle school students with educational disabilities, but those for 
academic and affective engagement were practically significant, with a small effect size (f:S > .10) 
(Table 18). 


Table 18 


The Relationship between UDL Implementation and Student Engagement, 
by Type of Engagement for Students with Educational Disabilities in Grades 6-8 



Academic 
(A = 134) 

Affective 
(N= 138) 

Cognitive 
(N= 138) 

Project school: B (SE) 

0.10(0.08) 

0.16 (0.12) 

-0.04 (0.13) 

Project school: P 

0.10 

0.11 

-0.02 

Model fit: F (df) 

3.11 *(3) 

3.76*(3) 

5.18** (2) 

Model fit: adjusted R2 

.26 

.06 

.06 


*p < .05, **p < .01, ***p < .001. 


Consistency . The findings about UDL implementation and engagement for all middle 
school students were not consistent with the findings for each student subgroup (Table 19). The 
relationship between UDL implementation and engagement was statistically significant for all 
students for each type of engagement, but the only statistically significant relationship for a 
student subgroup was for cognitive engagement, among English language learners. It is worth 
noting that sample sizes influence significant testing, such that differences among smaller 
samples, like the student subgroups, are less likely to be statistically significant. 

Table 19 


Summary of Significant Relationships between UDL Implementation and Student Engagement for All 
Students and Student Subgroups in Grades 6-8, by Type of Engagement 



All 

students 

Student subgroup 

English language learners 

Students with educational disabilities 

Type of 

Statistically 

Practically 

Statistically 

Practically 

Statistically 

Practically 

engagement 

significant 

significant 

significant 

significant 

significant 

significant 

Academic 

Y "k 

No 

No 

Small 

No 

Small 

Affective 

Y "f* "k 

Small 

No 

Small 

No 

Small 

Cognitive 

Yes** 

Small 

Yes* 

Medium 

No 

No 


*p < .05, **p < .01, ***p < .001. 


With respect to practical significance, only the relationship for affective engagement was 
consistent; it was significant with a small effect size for all students and also for each subgroup. 
For academic engagement the relationship was not practically significant for all students, but it 
was, with a small effect size, for both subgroups. Lastly, the relationship for cognitive 
engagement was practically significant for all students, with a small effect size and for one 
subgroup, English language learners, with a medium effect size. 

Independence in Learning Processes. The survey items about cognitive engagement 
also reflected independence in learning processes (see Table 2). The above findings indicate that 
independence in learning processes were significantly higher, both statistically and practically, at 
project schools for all students and also for English language learners, but not for students with 
educational disabilities. 
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Summary 

There was evidence for positive impacts from implementation of UDL practices on student 
processes, with variations by grade level, type of student process, and student subgroup. 

Based on classroom observations at UDL project schools, academic engagement, cognitive 
engagement, and independence in learning were all high for students in kindergarten-Grade 2, 
but affective engagement was not. Due to the limitations of observations (as noted above), it was 
not possible to analyze whether the findings were consistent for student subgroups. 

Students in Grades 3-8 at both UDL project and comparison schools completed surveys about 
their levels of engagement; the items on cognitive engagement also reflected independence in 
learning. For all students in Grades 3-5, affective engagement was significantly higher at UDL 
project schools than at comparison schools, but there were no differences for academic or 
cognitive engagement. Further, these findings were the same for both subgroups of students: 
English language learners and students with educational disabilities. For students in Grades 6-8, 
all three forms of engagement were significantly higher at UDL project schools than at 
comparison schools. These findings for all students were not the same as the findings for the two 
subgroups. 

Findings for Question 4 

What is the impact of UDL implementation on lesson planning for accessibility and teacher 
collaboration? 

To evaluate the impact of UDL implementation on teaching practices, data were collected 
through a staff survey of teachers (including media specialists). The results are based on 
responses from 40 staff members in 22 elementary and 18 middle schools. The results from the 
two school levels were similar and therefore were combined, with a note for any significant 
differences. 

Lesson Planning for Accessibility 

As discussed above, lesson planning for accessibility requires that teachers front-load and build 
the abilities of their students to make choices related to learning. Lront-loading means that the 
teacher incorporates UDL strategies during the creation of instruction and assessments, instead 
of adjusting them afterwards. Lurther, because UDL practices encourage students to select 
materials, tools, and products, teachers need to develop skills and routines that build each 
student’s ability to make choices that best support his/her learning. 

Front-loading. To evaluate whether teachers at project schools were front-loading, 
survey responses about the use of individual planning time were analyzed. Respondents reported 
to what extent they had changed the way they use their individual planning time in four areas, 
since their involvement with the UDL project (Table 20). Lor each area, about two thirds or 
more (63-73%) of the 40 respondents indicated that they completed changed or changed a lot. 
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Table 20 


Extent of Changes to Individual Planning Time by Area ( N = 40) 


Since your involvement with the UDL 
project, to what extent have you changed the 
way you use your individual planning time to: 

Completely 

changed 

Changed 
a lot 

Changed a 
moderate 
amount 

Changed 
a little 

No change 
at all 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

Plan your presentation to students 

2 

5 

23 

58 

ii 

28 

4 

10 

0 

0 

Plan materials for students 

4 

10 

22 

55 

14 

35 

0 

0 

0 

0 

Plan for student learning needs 

4 

10 

22 

55 

11 

28 

2 

5 

1 

3 

Plan student assignments 

5 

13 

24 

60 

8 

20 

0 

0 

3 

8 


Further, in open ended questions, respondents described the ways that their individual planning 
time had changed for each of the four areas: presentation to students, materials for students, 
student learning needs, and student assignments. Responses across all four areas were analyzed 
for evidence that teachers incorporated UDL strategies during their planning time (Table 21). 
About two thirds of the respondents said that in their individual planning time they now build in 
student choices (26 of 40, 65%) and incorporate more visuals, images, and graphic organizers 
(25 of 40, 63%). Furthermore, at least one half of the respondents stated they now consider 
student barriers and learning styles (23 of 40, 58%), incorporate digital media and technology 
(22 of 40, 55%), and provide a variety of formats beyond text and speaking (20 of 40, 50%). 
Examples of these UDL-related changes are found in the Appendix. 


Table 21 

UDL Related Changes to Individual Planning Time for 
Presentations, Materials, Learning Needs, and Assignments ( N = 40) 


Changes to planning time (open-ended) 

n 

% 

Build in student choices 

26 

65 

Incorporate (more) visuals/images/graphic organizers 

25 

63 

Consider predictable student barriers/student needs/learning 
styles/successful options 

23 

58 

Incorporate digital media/technology 

22 

55 

Provide additional formats beyond viewable text plus teacher 
speaking/variety of ways/multiple explanatory devices 

20 

50 

Options where students can demonstrate understanding 

16 

40 

More interactive/hands-on/less worksheets 

10 

25 

Reflective/have students think about choice 

3 

8 

Different ways to assess 

3 

8 

Other UDL comments 

10 

25 

No comment 

1 

3 


Note. Each respondent’s answer could include more than one category. 


Significant differences, shown in Table 22, were found between school levels for two changes. 
More elementary (12 of 22, 55%) than middle school (4 of 18, 22%) respondents indicated 
options where students can demonstrate understanding (y 2 (df = 1) = 24. 1, p < .05). Also, more 
elementary (9 of 22, 41%) than middle school respondents (1 of 18, 6%) included other UDL 
comments (% 2 (df = 1) = 24. 1 ,p< .05). 
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Table 22 

Selected UDL Related Changes to Individual Planning Time by School Level 



School level 


Elementary 

Middle 


(n = 22) 

(n = 18) 

Changes to planning time (open-ended) 

n % 

n % 

Options where students can demonstrate understanding 

12 55 

4 22 

Other UDL comments 

9 41 

1 6 


Note. Each respondent’s answer could include more than one category. 


Most teachers were front-loading, as indicated by the above responses. First, about two thirds of 
respondents completely changed or changed a lot their use of individual planning time, since 
involvement with the UDL project. Second, more than one half of respondents described these 
changes in terms of key UDL strategies: building in student choices, considering student barriers 
and learning styles, or providing a variety of formats beyond text and speaking. 

Building Student Choice. As described above, UDL principles emphasize providing 
choices to students in how they gain information and show what they know. Therefore, teachers 
reported on whether they had a plan to build students’ ability to make choices throughout this 
school year (Table 23). Almost three fourths of the participants (29 of 40, 73%) replied that they 
have an informal plan. Only 4 of 40 (10%) teachers indicated that they have a formal plan, and 
7 (17%) replied that they have no plan or did not respond. 

Table 23 

Types of Plans to Build Students’ Ability to Make Choices (N = 40) 


Did you have a plan to build students’ ability to make choices 
throughout this school year? 

n 

% 

Yes, an informal plan 

29 

73 

Yes, a formal plan 

4 

10 

No plan 

6 

15 

No answer 

1 

2 


Additionally, teachers were asked to describe their plan for building student choice throughout 
the school year in an open-ended question. However, 10 of the 33 teachers (30%) with a plan 
did not give a description (Table 24). Among the 23 respondents who did describe their plan, 
about one half (11 of 23, 48%) indicated that students choose their best way to learn; examples 
of these plans are in the Appendix. 
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Table 24 


Description of Plan for Building Student Choice Throughout the School Year 



Respondents 

Respondents who 


with a plan 

described 

their plan 

Plan for building student choice throughout 

(N = 

33) a 

(N = 

23) b 

this school year (open-ended) 

n 

% 

n 

% 

Students choose their best way to learn 

ii 

33 

ii 

48 

Students reflect on choices 

5 

15 

5 

22 

Model choices 

4 

12 

4 

17 

Gradually introduce choices 

4 

12 

4 

17 

Other choice comments 

4 

12 

4 

17 

Other comments 

2 

6 

2 

9 

No comment 

10 

30 

NA 

NA 


Note. Respondent’s answers could include more than one category. 

^Includes 29 respondents with an informal plan and 4 respondents with a formal plan. 
Includes 19 respondents with an informal plan and 4 respondents with a formal plan. 


Teacher Collaboration 

As discussed above, teachers involved in the UDL projects should collaborate in several ways. 
They should discuss student learning profiles with colleagues to better understand how to ensure 
access to each student. Also, given the important role of student choice (i.e., among learning 
tools, tasks, or products) within UDL, teachers should work together to build students’ capacity 
for making choices throughout the school year and across grades. Lastly, given the need to 
create accessible materials as part of UDL practices, teachers are expected to share responsibility 
for creating and sharing them. 

Discussion of Student Learning and Building Student Choice. Teachers reported how 
often they discussed student learning preferences with other teachers during this school year. 
The majority, (24 of 40, 61%), said a few times a week or a few times a month (Table 25). 
Further, teachers reported on their collaboration to build students’ capacity for making choices. 
About one half of respondents (21 of 40, 53%) said they discussed building student choice with 
other teachers of the same grade level a few times a month or a few times a week. However, 
discussions with other teachers across grade levels were less frequent; only one fifth (8 of 40, 
20%) of respondents reported doing so a few times a month or a few times a week. 

Table 25 

Frequency of Discussing Student Learning and Student Choice ( N = 40) 


3-4 1-2 

A few times A few times About once times a times a Never or 
During this school year, a week a month a month year year no answer 


how often did you discuss. . . 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

Student learning preferences 
with other teachers 

5 

13 

19 

48 

5 

13 

4 

10 

i 

2 

6 

15 

Building student choice 
with other teachers for the 
same grade level 

8 

20 

13 

33 

6 

15 

5 

13 

i 

2 

7 

17 

Building student choice 
with other teachers across 
grade levels 

1 

2 

7 

18 

9 

23 

8 

20 

8 

20 

7 

17 
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Additional survey items about discussions regarding students concerned team planning time; 
teachers reported on changes in using that time to discuss students, since their involvement with 
the UDL project (Table 26). No one indicated a complete change and almost one fourth (8 of 33, 
24%) reported no change. However, just over one half (18 of 33, 54%) replied that their team 
planning time has changed a lot or a moderate amount in this aspect. Similarly, among the 
11 teachers that reported planning with a second team, just over one half (6 of 11, 54%) replied 
that their time has changed a lot or a moderate amount with respect to discussing students. 


Table 26 

Extent of Changes to Team Planning Time to Discuss Students 


Since your involvement with the UDL project, to 
what extent have you changed the way you use 

Completely 

changed 

Changed 
a lot 

Changed a 
moderate 
amount 

Changed 
a little 

No 

change 
at all 

your team planning time. . . 

n % 

n % 

n % 

n % 

n % 

To discuss students. (N = 33) a 

0 

0 

8 24 

10 

30 

7 

21 

8 24 

With a second team to discuss students. (N = 1 l) b 

0 

0 

2 18 

4 

36 

0 

0 

5 46 


^Includes only the 33 respondents who answered this question. 
Includes only the 1 1 respondents who answered this question. 


When asked in an open-ended question, to describe changes to their team planning time to 
discuss students, more than one third (9 of 25, 36%) of those who responded mentioned that they 
discuss different learning styles or how all students can access the curriculum (Table 27). 

Table 27 


Description of Changes to Team Planning Time to Discuss Students ( N = 25) 


Change (open-ended) 

n 

% 

Discuss different learning/all students access to curriculum /how to 
address barriers 

9 

36 

Discuss various ways to present 

3 

12 

Plan choices 

3 

12 

Plan to increase student engagement 

2 

8 

Discuss strategies and interventions 

2 

8 

Other changes 

4 

16 

Not much change 

2 

8 

Other comments 

2 

8 


Note. Includes only respondents who indicated they had changed their team planning time for 
discussing students. Each respondent’s answer could include more than one category. 


Materials That Enhance Accessibility . Survey respondents described in their own words 
to what extent they did each of the following: work with other teachers to create materials that 
enhance accessibility, share materials with teachers in other content areas, and share materials 
with teachers in other grade levels. Their responses were classified into one of three categories: 
frequently/regularly, somewhat, rarely/not often. 

About one half (23 of 40, 57%) of the respondents indicated that they frequently or regularly 
work with other teachers to create materials that enhance accessibility (Table 28). Close to one 
half (18 of 40, 45%) indicated that they frequently or regularly share materials with teachers in 
other content areas. Only one fifth (8 of 40, 20%) indicated that they frequently or regularly 
share materials with teachers in other grade levels; close to one third share with teachers in other 
grade levels either somewhat (13 of 40, 32%) or rarely (12 of 40, 30%). In an open-ended 
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question, respondents described their work with other teachers on materials that enhance 
accessibility; examples of verbatim comments from those who regularly work with other 
teachers are in the Appendix. 


Table 28 


Extent of Working with Other Teachers to Create or Share Materials that Enhance Accessibility (N = 40) 


To what extent do you. . . 

Frequently/ 

regularly 

Somewhat 

Rarely/not often/ 
not applicable 

No 

answer 

n 

% 

n 

% 

n 

% 

n 

% 

Work with other teachers to create materials 

23 

57 

7 

18 

4 

10 

6 

15 

Share materials with teachers in other content areas 

18 

45 

7 

18 

9 

22 

6 

15 

Share materials with teachers in other grade levels 

8 

20 

13 

32 

12 

30 

7 

18 


Note. Responses were classified into frequency categories. 


To share materials that enhance accessibility, schools should have electronic folders with these 
materials. Survey respondents reported on whether already-created materials that enhance 
accessibility for students are available electronically at their school. The majority of respondents 
(28 of 40, 70%) replied that these materials are available and that they know how to get them 
(Table 29). 

Table 29 


Availability of Electronic Already-created Materials ( N = 40) 


At your school, are already-created materials that enhance accessibility for 
students available electronically? 

n 

% 

Yes, and I know how to get these materials. 

28 

70 

Yes, but 1 don’t know how to get these materials. 

1 

2 

No, my school does not have these materials available electronically. 

3 

8 

1 don’t know whether these materials are available electronically at my school. 

2 

5 

No answer. 

6 

15 


Among the 28 respondents who said materials were available electronically and who knew how 
to access them, more than three fourths (22 of 28, 79%) strongly agreed or agreed that they 
check for these materials when planning a lesson (Table 30). Close to two thirds (18 of 28, 64%) 
strongly agreed or agreed that they regularly add to the materials that are available electronically. 


Table 30 

Usage of Already-created Materials That Enhance Accessibility ( N = 28) 



Strongly 

agree 

Agree 

Neither 
agree nor 
disagree 

Disagree 

Strongly 

disagree 

Survey item 

n % 

n 

% 

n 

% 

n 

% 

n % 

When planning a lesson, I always check for already- 
created materials that enhance accessibility for 
students and are available electronically at my school. 

5 

18 

17 

61 

4 

14 

2 

7 

0 

0 

I regularly add to the materials that enhance 
accessibility for students and that are available 
electronically at my school. 

4 

14 

14 

50 

3 

11 

7 

25 

0 

0 


Note: Includes only the 28 respondents who said materials were available electronically and who knew how to access them. 
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Summary. The teacher practices that are key to UDL implementation are lesson planning 
for accessibility and teacher collaboration. Overall, there was evidence for positive impacts of 
UDL implementation on these teacher practices. 

Lesson planning for accessibility requires front-loading, which means that the teacher 
incorporates UDL strategies during the creation of instruction and assessments. Most survey 
respondents were front-loading, as indicated by the following. First, since their involvement 
with the UDL project, about two thirds of respondents reported they had “completely changed” 
or “changed a lot” their use of individual planning time. Second, more than one half of 
respondents described these changes to their planning time in terms of key UDL strategies: 
building in student choices, considering student barriers and learning styles, or providing a 
variety of formats beyond text and speaking. 

Lesson planning for accessibility also requires building student choice throughout the school 
year. Most survey participants made plans to build student choice throughout the school year. 
More than one half of the respondents described such a plan, typically an informal one. 

Teacher collaboration focuses on discussions with other teachers about students and student 
choice and on materials that enhance accessibility. Among survey respondents, more than one 
half had discussions with other teachers at least a few times a month about student learning 
preferences and about building student choice within their grade level. Most responding teachers 
shared responsibility for creating materials that enhance accessibility. About one half of 
respondents indicated that they frequently or regularly work with other teachers to create such 
materials that enhance accessibility and regularly add them to a shared electronic folder at their 
school. Further, the majority of respondents reported that they knew how to access this 
electronic folder and checked this folder for materials when planning a lesson. 

However, teacher collaboration across content areas or grade levels was less frequent. Less than 
one half of survey respondents frequently or regularly shared materials that enhance accessibility 
with teachers in other content areas. Only one fifth of respondents frequently or regularly 
discussed students or shared materials that enhance accessibility across grade levels. 
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Discussion 

The goals of the UDL projects include the following: to identify models for schools to develop 
leadership teams focused on schoolwide implementation of UDL over a three- to five-year 
period and to identify scalable models to guide the professional learning of UDL leadership 
teams across multiple schools. The findings from this evaluation suggest that the model used in 
the UDL pilot projects was successful; there was evidence for positive impact on teachers’ 
implementation of UDL practices, on the student processes of engagement and independence in 
learning, and on teacher practices of lesson planning for accessibility and collaboration. At least 
one half of teachers consistently implemented UDL or reported using the key teacher practices; 
this moderate level of implementation across all teachers is reasonable, given that the evaluation 
occurred two to three years into a five year project. 

The purpose of the observations was not to evaluate individual teachers (because only one class 
was observed) but to evaluate the implementation level of UDL instructional practices within 
project schools. Given that implementation was at a moderate level for the project, it was 
assumed that project teachers used UDL practices more frequently than teachers at comparison 
schools. Therefore, the above analyses of student engagement included students of all project 
teachers. 

However, it is possible that the project teachers with inconsistent implementation of UDL did not 
increase the engagement of their students and thus should be excluded from analysis. Therefore, 
the analyses of student engagement were redone using only the students of project teachers with 
consistent implementation (along with their comparison classrooms). The results were very 
similar to those presented above, in terms of higher engagement among students in project 
schools. Re-analysis of subgroups was not possible, due to the very small sizes of these groups. 

As noted above, consistent implementation of UDL practices was lower among middle school 
than elementary school teachers. However, the apparent impact of UDL on student processes 
was greater among the older students. While each type of engagement differed significantly 
between project and comparison schools for Grades 6-8, only affective engagement differed 
significantly for students in Grade 3-5. A possible explanation is that the engagement level was 
so high among all the younger students, that differences between the two groups of schools were 
less likely. The mean value of each type of engagement was closer to the scale’s maximum for 
the younger students than for the older ones (across all schools) (Table 31). For academic 
engagement, the mean for Grades 3-5 was 3.43, which is closer to the scale’s maximum value of 
4.0, than the mean of 3.10 for Grades 6-8. Likewise, for affective engagement, the mean of 3.34 
for Grades 3-5 was closer to the scale’s maximum value of 4.0, than the mean of 2.81for the 
older students. Finally, for cognitive engagement, the mean of 3.81 for the younger students was 
closer to the scale’s maximum of 5.0, than the mean of 3.42 for Grades 6-8. 

Table 31 


Mean and Standard Deviation of Engagement Scales by School Level 


Type of engagement (range) 


Grades 3-5 

Grades 6-8 

n 

Mean 

Standard deviation 

n 

Mean 

Standard deviation 

Academic (1. 0-4.0) 

565 

3.43 

0.48 

811 

3.10 

0.56 

Affective (1. 0-4.0) 

566 

3.34 

0.60 

811 

2.81 

0.70 

Cognitive (1. 0-5.0) 

523 

3.81 

0.76 

810 

3.42 

0.82 
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Recommendations 

Based on findings from the evaluation, we recommend that program staff focus training, 

guidance, and other supports on the following suggestions for improving future implementation 

ofUDL: 

• Encourage leadership at all schools to increase the number of UDL practices implemented by 
providing additional training, Professional Learning Communities, and collaborative 
planning to promote use of the following practices: choices of products, choices of responses, 
variety of formats for handouts, and reflection about choice. 

o Just over one half of teachers had consistent implementation, which meant using at 
least four of the nine UDL practices. 

• Encourage leadership at middle schools to increase the number of UDL practices 
implemented by providing additional training, Professional Learning Communities, and 
collaborative planning to promote use of stations/centers and routines for making choices. 

o Consistent implementation of UDL practices was lower among middle school 
teachers than among elementary school teaches. 

• Consider encouraging teachers to formally plan how they will build students’ abilities to 
make choices throughout the school year; provide specific models and templates that teachers 
can use. 

o Most teachers indicated that they had an informal plan to build students’ abilities to 
make choices, but not a formal plan. 

• Increase the sharing of materials that enhance accessibility with teachers in other content 
areas and across grade levels; provide specific ways and examples for teachers to use. 

o Less than one half of respondents regularly shared materials with teachers in other 
content areas and only one fifth shared materials across grade levels. 
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Appendix: Detailed Responses from Staff Survey 

Table A1 


Examples of Changes to Individual Planning Time 


Category 

Examples 

Build in student 
choices 

• I try to make sure there are more materials for student choice. 

• Embedding choice into how to learn and how to demonstrate knowledge 

• I try to build in choices for students whenever I can. This will be an on- 
going process. 

• I now, more often, give students several options for responding to 
assignments. For example, in my reading class, students show mastery of 
vocabulary by drawing, acting it out, or writing sentences. 

Incorporate (more) 

visuals/images/graphic 

organizers 

• Consistently adding visuals to flipcharts embedding videos 

• I look for more visuals and more interesting ways to present through 
visuals and video clips 

• Much of the multiple modes of presenting information include visuals, so 
much of my planning time is spent creating these. 

Consider predictable 
student 

barriers/student 

needs/learning 

styles/successful 

options 

• Before I present, I take into consideration the different ways students 
learn information. I don't present the same way all the time. 

• More consciously thought about "who is being left out?" and "what can 
provide to help those students overcome their barriers to learning?” 

• I've spent a lot more time thinking and planning for specific student 
challenges. I've found that when I design the lesson thinking of these 
students from the beginning, the entire lesson works better for all 
students. 

Incorporate digital 
media/technology 

• Presentations have change to consistently provide various ways to present 
information to students. This includes utilizing various components of 
Flipcharts and embedding videos, images and sound to present the 
information. 

• I am always looking for digitized text so that the students can have a copy 
that was properly manipulated. 

• Offering students choice as far as the way they would like to present their 
learning both through the use of technology or without technology. Also, 
different ways to incorporate technology (i.e. Word Q and Natural 
Reader) to provide accommodations. 

Provide additional 
formats beyond 
viewable text plus 
teacher 

speaking/variety of 
ways/multiple 
explanatory devices 

• Making sure that there are a variety of ways for the students to 
understand the information... verbal, visual, hands on, small group, large 
group, student discussions, etc. 

• I look for many different modalities- using the promethean flipcharts, 
music, kinesthetic learning etc. 

• Simply taking additional time to incorporate a variety of learning 
approaches, particularly in math and reading. Always plan for rotations in 
math and reading (almost daily). 
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Table A2 

Examples of Plans to Build Students’ Ability to Make Choices 


Category 

Examples 


• Students have the opportunity to complete written assignments on the 
computer, variety of paper styles, pens, colored pencils, markers, etc. 
At the beginning of the year I explained to all students that we are an 
UDL classroom and they have choices to decide on their own the best 
way they learn and to show what they know. 

Students choose their 
best way to learn 

• I explain to students that 1 give them choices. They are to choose 
according to their likes, strengths, and opportunities for challenge. 
What they choose should have an end product that will show their 
best effort. 


• Students were taught how to recognize their own learning strengths 
and needs. They were then encouraged to think about the choices 
they were given so they could make the correct choice. Often, if they 
didn't do well, I would encourage them to make a different choice 
next time so they could find the method that worked best for them. 

Table A3 

Examples of Collaboration to Create or Share Materials That Enhance Accessibility 


Category Examples 

• 1 work with teammates on a weekly basis to plan lessons with 
accessibility in mind. Sometimes we will take materials and resources 
from the online curriculum and modify it to meet the needs of our 
learners. Other times, we will take materials from previous years to do 
the same or create materials ourselves that attempts to account for 

Frequently/regularly work student barriers to learning. 

with other teachers to • There was lots of collaboration among teammates and members of the 

create materials that UDL team. Teammates met regularly each week, and UDL members 

enhance accessibility met each month to share and create materials. 

• 1 attended team meetings at each grade level on a regular basis and 
sparked many conversations about how to make lessons more UDL. 1 
created graphic organizers, flipcharts, project choices for various 
teams. 1 meet with teams/teachers informally everyday but formally 
about every 2 weeks. 

We use t-share and a calendar with hyperlinks daily. 

Very often, send emails to grade level team and specialists frequently 
to share created organizers, flipcharts, etc. occasionally uploaded 
resources to county websites. 

Every day. Whatever I make and find each day is either emailed 
directly to my team or put in Teacher Shared for others to use. 
Frequently/regularly share • Weekly and as needed 
materials with teachers in • Regularly with entire school 

other grade levels • Materials created go into the teacher shared folder 


Frequently/regularly share 
materials with teachers in 
other content areas 
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